SRE-3988 ci: ucx-devel is used for mercury-ucx build. by grom72 · Pull Request #18949 · daos-stack/daos

grom72 · 2026-08-27T11:01:41Z

UCX is no longer built from source by default, the distribution provided packages are used instead.

Requires:

Steps for the author:

Commit message follows the guidelines.
Appropriate Features or Test-tag pragmas were used.
Appropriate Functional Test Stages were run.
At least two positive code reviews including at least one code owner from each category referenced in the PR.
Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

Gatekeeper requested (daos-gatekeeper added as a reviewer).

github-actions · 2026-08-27T11:01:58Z

Errors are Unable to load ticket data
https://daosio.atlassian.net/browse/SRE-3988

daosbuild3 · 2026-08-27T12:09:24Z

Test stage Unit Test with memcheck completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18949/1/testReport/

daosbuild3 · 2026-08-27T18:52:23Z

Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18949/6/testReport/

daosbuild3 · 2026-08-29T10:25:52Z

Test stage Test RPMs on Leap 15 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18949/11/execution/node/610/log

UCX is no longer built from source, the distribution provided packages are used instead Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

Treat an absent ConnectX adapter as an expected condition during provisioning. Prevent the probe from triggering the ERR trap when no Mellanox hardware is present. Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com> Priority: 2

daltonbohning · 2026-09-01T20:15:51Z

-    reqs.define('ucx',
-                retriever=GitRepoRetriever(),
-                libs=['ucs', 'ucp', 'uct'],
+    # UCX is no longer built from source, the distribution provided packages are used instead.


Nit - IMO the comment should not say that UCX is "no longer built from source", but rather just state that it comes from the distribution. Because once this is landed, it is no longer a "change", it just simply "is".

we need to keep that capability though for testing and debugging purposes so we can't just simply remove all that code. We need 3 things:

be able to use the DOCA OFED UCX rpms

be able to compile UCX from source

be able to use an externally installed UCX path (hence why we need a single UCX package and not split them up as this was done in that PR)

daltonbohning · 2026-09-01T20:20:52Z

                libs=['mercury'],
                pkgconfig='mercury',
-                requires=['boost', 'ofi', 'ucx'] + libs,
+                requires=['boost', 'ofi', 'ucx_ucs', 'ucx_ucp', 'ucx_uct'] + libs,


We will need a way to conditionally build without UCX because we need to be able to build DAOS from source on systems that do not have or use UCX. For example, on aurora.

yeah I have been working myself on adding a new option to be able to compile without UCX (out of scope of that PR). I don't think we should be separating UCX dependencies though. It should remain a single UCX package that has 3 libraries.

soumagne · 2026-09-01T20:25:13Z

                libs=['mercury'],
                pkgconfig='mercury',
-                requires=['boost', 'ofi', 'ucx'] + libs,
+                requires=['boost', 'ofi', 'ucx_ucs', 'ucx_ucp', 'ucx_uct'] + libs,


yeah I have been working myself on adding a new option to be able to compile without UCX (out of scope of that PR). I don't think we should be separating UCX dependencies though. It should remain a single UCX package that has 3 libraries.

soumagne · 2026-09-01T20:26:14Z

 mercury=https://github.com/mercury-hpc/mercury.git
 protobufc=https://github.com/protobuf-c/protobuf-c.git
-ucx=https://github.com/openucx/ucx.git



we need to keep that in and still be able to build UCX from source when needed.

The problem I have here is that we keep some configuration that will not be verified as soon as we start using OS provided ucx-devel in our CI.
Leaving as it is can be misinformative giving the impression that DAOS uses/requires this particular version of UCX, what is not true.

soumagne · 2026-09-01T20:29:34Z

-    reqs.define('ucx',
-                retriever=GitRepoRetriever(),
-                libs=['ucs', 'ucp', 'uct'],
+    # UCX is no longer built from source, the distribution provided packages are used instead.


we need to keep that capability though for testing and debugging purposes so we can't just simply remove all that code. We need 3 things:

be able to use the DOCA OFED UCX rpms

be able to compile UCX from source

be able to use an externally installed UCX path (hence why we need a single UCX package and not split them up as this was done in that PR)

soumagne · 2026-09-01T20:33:02Z

    libtool \
+    libucp-devel \
+    libucs-devel \
+    libuct-devel \


we should not be installing ucx everywhere either in my opinion unless the CI is requesting UCX testing. Maybe we need to revisit that once I've added the option to selectively compile UCX...

The first goal of install-*.sh scripts is to setup DAOS development environment.
We need to install ucx-devel to be able to compile mercury without the need of building ucx from scratch every time.

Someone must uninstall ucx-devel (libucp-devel,...) first, if he wants to build ucx from source code.

ok fair enough

This reverts commit bf74dca. Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

By default the mercury package is build against UCX package(s) delivered by OS. Uninstal ucx-devel/libucp-devel,libucs-devel,libuct-devel/libucx-dev to build UCX from source. Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com> Priority: 2

grom72 · 2026-09-02T14:35:53Z

@daltonbohning @soumagne
The new approach is to install ucx-devel RPM by default.
Developer must uninstall it if wants to use other version (source or DOAC)

soumagne

I think we can try this approach, that looks reasonable to me. Thanks!

grom72 changed the title ~~Grom72/sre 3988 separate ucx lib~~ SRE-3988 ci: separate ucx lib Aug 27, 2026

grom72 force-pushed the grom72/SRE-3988-separate-ucx-lib branch 2 times, most recently from 910691c to 5747dee Compare August 28, 2026 19:22

grom72 force-pushed the grom72/SRE-3988-separate-ucx-lib branch 2 times, most recently from 8d69923 to 99af536 Compare September 1, 2026 14:45

grom72 changed the title ~~SRE-3988 ci: separate ucx lib~~ SRE-3988 ci: ucx-devel is used for mercury-ucx build. Sep 1, 2026

grom72 added 2 commits September 1, 2026 22:05

SRE-3988 ci: Use ucx-devel from OS RPMs

bf74dca

UCX is no longer built from source, the distribution provided packages are used instead Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

Remove obsolete suppression

b58ffbd

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

grom72 force-pushed the grom72/SRE-3988-separate-ucx-lib branch from 2086d98 to 5b8cebe Compare September 1, 2026 20:06

grom72 force-pushed the grom72/SRE-3988-separate-ucx-lib branch from 5b8cebe to bff75f6 Compare September 1, 2026 20:07

grom72 marked this pull request as ready for review September 1, 2026 20:07

grom72 requested review from a team as code owners September 1, 2026 20:07

grom72 requested review from JohnMalmberg, daltonbohning and janekmi September 1, 2026 20:07

daltonbohning requested changes Sep 1, 2026

View reviewed changes

soumagne requested changes Sep 1, 2026

View reviewed changes

soumagne reviewed Sep 1, 2026

View reviewed changes

grom72 added 2 commits September 2, 2026 08:45

Revert "SRE-3988 ci: Use ucx-devel from OS RPMs"

fd78963

This reverts commit bf74dca. Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

grom72 requested review from daltonbohning and soumagne September 2, 2026 14:29

soumagne approved these changes Sep 2, 2026

View reviewed changes

daltonbohning approved these changes Sep 2, 2026

View reviewed changes

daltonbohning added the release-3.0.0 Targeted for release 3.0.0 label Sep 2, 2026

grom72 requested a review from a team September 3, 2026 14:40

daltonbohning merged commit 79703fc into master Sep 3, 2026
56 of 57 checks passed

daltonbohning deleted the grom72/SRE-3988-separate-ucx-lib branch September 3, 2026 15:11

Uh oh!

Conversation

grom72 commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Steps for the author:

After all prior steps are complete:

Uh oh!

github-actions Bot commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daosbuild3 commented Aug 27, 2026

Uh oh!

daosbuild3 commented Aug 27, 2026

Uh oh!

daosbuild3 commented Aug 29, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

grom72 Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

grom72 commented Sep 2, 2026

Uh oh!

soumagne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

grom72 commented Aug 27, 2026 •

edited

Loading

github-actions Bot commented Aug 27, 2026 •

edited

Loading

grom72 Sep 1, 2026 •

edited

Loading